SEA-PARAM: Exploring Schedulers in Parametric MDPs
نویسندگان
چکیده
We study parametric Markov decision processes (PMDPs) and their reachability probabilities ”independent” of the parameters. Different to existing work on parameter synthesis (implemented in the tools PARAM and PRISM), our main focus is on describing different types of optimal deterministic memoryless schedulers for the whole parameter range. We implement a simple prototype tool SEA-PARAM that computes these optimal schedulers and show experimental results.
منابع مشابه
Time-Bounded Reachability in Distributed Input/Output Interactive Probabilistic Chains
We develop an algorithm to compute timed reachability probabilities for distributed models which are both probabilistic and nondeterministic. To obtain realistic results we consider the recently introduced class of (strongly) distributed schedulers, for which no analysis techniques are known. Our algorithm is based on reformulating the nondeterministic models as parametric ones, by interpreting...
متن کاملPARAM: A Model Checker for Parametric Markov Models
We present PARAM, a model checker for parametric discretetime Markov chains (PMCs). PARAMcan evaluate temporal properties of PMCs and certain extensions of this class. Due to parametricity, evaluation results are polynomials or rational functions. By instantiating the parameters in the result function, one can cheaply obtain results for multiple individual instantiations, based on only a single...
متن کاملA Simple and Efficient Statistical Model Checking Algorithm to Evaluate Markov Decision Processes
We propose a simple and efficient technique that allows the application of statistical model checking (SMC) to Markov Decision Processes (MDP). Our technique finds schedulers that transform the original MDP into a purely stochastic Markov Chain, on which standard SMC can be used. A statistical search is performed over the set of possible schedulers to find the best and worst with respect to the...
متن کاملA Model Based Method for Characterization and Location of Curved Image Features
This report deals with the development of a parametric model based method to locate and characterize precisely important curved features such as ellipses and B-splines based curves. The method uses all the grey level information of the pixels contained within a window around the feature of interest and produces the complete parametric model that best approximates in a mean-square sense the obse...
متن کاملExtending the Bellman equation for MDPs to continuous actions and cont. time in the discounted case
Recent work on Markov Decision Processes (MDPs) covers the use of continuous variables and resources, including time. This work is usually done in a framework of bounded resources and finite temporal horizon for which a total reward criterion is often appropriate. However, most of this work considers discrete effects on continuous variables while considering continuous variables often allows fo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2017